fix schedule_downstream_tasks bug (#42582) #43299
Conversation
* fix schedule_downstream_tasks bug * remove partial_subset * Update comment --------- Co-authored-by: 维湘 <[email protected]> (cherry picked from commit 3fceaa6)
|
This is backport of #42582 Since we are closing to 2.10.3 I merged #42582 as I believe it solves often occuring "serialization" issues (with partial_subset) that happen in "mini-scheduler" and causes failures of celery workers - and tests had shown that partial_subset are also wrong when it comes to determining eligibility of dependent tasks in some cases. @uranusjr and @ashb -> if you have any concerns about it this is the time to raise it before we merge - I think you were mostly those who knew most about using |
potiuk
requested review from
uranusjr,
shahar1,
jedcunningham,
ephraimbuddy and
utkarsharma2
|
The failure here should be fixed by #43307 |
|
Failing tests fixed in tip of |
* fix schedule_downstream_tasks bug * remove partial_subset * Update comment --------- Co-authored-by: 维湘 <[email protected]> (cherry picked from commit 3fceaa6) Co-authored-by: luoyuliuyin <[email protected]>
fix schedule_downstream_tasks bug
remove partial_subset
Update comment
Co-authored-by: 维湘 [email protected]
(cherry picked from commit 3fceaa6)
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.